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ASSESSMENT OF THE 
ENGLISH LANGUAGE ARTS COMMON CURRICULUM GOALS 



Report 2: Speaking Skills 



School districts in Oregon are required in Standards for Public Eleinentary and Secondary 
Schools 581-22-602 and -606 to use student assessment information on the Common 
Curriculum Goals to assist in making decisions about instruction of individual students 
and effectiveness of instructional programs. The standards suggest that a broad range of 
information is required to profile student and program progress and needs. There is also 
the assumption that instructional programs in schools have a clear alignment among the 
goals for instruction, the activities in the classroom, and the assessment of students' 
knowledge and skill. 

This report provides suggestions on how a language arts speaking assessment program 
might be structured to ensure that districts carry out the intent of the state standards. 

The suggestions oifered within this report are based on what current research indicates 
works best in measuring speaking. Cle'^.rly, there may be differences in speaking 
assessment district to district, and eve^i classroom to classroom within the same 
building. To the extent that classroom or proj?ram assessment approaches differ from 
what is suggested here, those differences should nevertheless reflect a sound research 
base. 

This report includes: 

1. A list of the Common Curriculum Goals that relate to speaking (keyed to the 
Essential Learning Skills). 

2. General implications for assessment. 

3. Criteria for differentiating among insufficient, acceptable and ideal assessment 
practices at the classroom and district levels. 

4. Bibliography of speaking assessment sources. 

5. Sample speaking assessment tools and procedures. 



COMMON CURRICULUM GOALS RELATED TO SPEAKING 

The following Common Curriculum Goals relate to speaking skills and include both formal 
and informal speaking as well as reasoning skills. Other Common Curriculum Goals may 
also be assessed through speaking activities (note particularly 2.4, 2.5, 2.8, 2.10). 
However, in order to avoid repetition in these reports, each Common Curriculum Goal 
appears only once in the area where it is iu.,st frequently and easily assessed. Where 
appropriate, the Common Curriculum Goals have been keyed to relevant Essential 
Learning Skills (ELS), which cut across curriculum areas. 

Some procedures and resources are included later in this report which may be helpful in 
assessing speaking skills. However, it is NOT necessary that these Common Curriculum 
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Goals be individually assessed, nor assessed separately within different content areas. In 
other words, a well-structured performance-based speaking assessment — as an 
exaniple — might well meet the assessment requirements for all the Common Curriculum 
Goals listed here. 

Further, districts that are focusing on the Essential Learning Skills may find creative 
ways to structure assessments that touch on more than one curriculum area: speech and 
writing, for example, or speech and math. This integrative approach is encouraged to the 
extent that districts find it a natural and logical outgrowth of their preferred assessment 
procedures; however, it is also perfectly acceptable for districts to assess different 
curriculum areas separately. 

Common Curriculum Goals (Relevant to Speaking) 

Students will: 

2.1 Speak and read orally with standard pronunciation, appropriate volume, rate, 
gestures and inflections (ELS 1.3). 

2.2 Use oral communication to give and receive information, directions, and for 
enjoyment (HLS 2.3). 

2.3 Use group discussion skills appropriately. 

2.6 Select and use language, gestures and symbols appropriate to audience, 
purpose, topic and setting when planning oral and written presentations (ELS 
5.5). 

2.15 Use oral communication to influence others and to respond to persuasion (ELS 
4.2). 

2.16 Demonstrate an appreciation of writing and oral communication skills as a 
lifelong means of self-expression, learning and personal development. 

2.20 Formulate and support a position orally and in writing using appropriate 
information and sound argument (ELS 6.5). 

2.21 Reflect upon and improve own reasoning in oral and written communications 
(ELS 6.6). 



IMPLICATIONS FOR ASSESSMENT 

The process of developing an oral presentation is similar to the process of writing. In the 
classroom setting, ceachers can assess student skills in both the development process and 
the final product. Student skills in group discussions and other everyday applications of 
speaking should also be evaluate J. 
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In assessing the development of a presentation, thi^ee stages may be examined — planning, 
practicing, and revising. In the planr.ing stages, teachers should review written materials 
such as note cards, outlines:, idea sheets, etc. as students design the presentation and plan 
for the audience (CCG 2.f). 

As students begin to formulate their presentations, obsei-vations can be made as they 
practice and receive feedback from small groups or partners on their pronuxiciation, 
volume, rate, language, gestures, and symbols (CCG 2.1, 2.2, 2.6). Sample 
peer-evaluation forms are included in the appendices. Some group discussion skills 
(CCG 2.3) may also be assessed at this stage. Judgments about revisions made in 
response to peer or teacher feedback may be ongoing when students practice and refine 
their presentations. 

The final presentation involves assessment of a variety of skills including language, 
deliver>% organization, and responsiveness to audience feedback, both verbal and 
nonverbal (CCG 2.1, 2.2, 2.6, 2.16). Depending on the type of presentation, it rray be 
possible to include judgments about persuasion (CCG 2.15) and argumentation or debate 
(CCG 2.20). Checklists or score sheets (see appendices) can be helpful evaluation 
instruments in addition to serving as recording tools for comments and feedback. 

When students engage in small and large group discussions, a variety of speaking skills 
may be assessed. It may be helpful to use a checklist (see appendices) or to collect 
student reactions to the discussion activity as part of the assessment. Potential areas 
include language, delivery, organization, group process skills, appreciation, persuasion, 
argumentation, and reasoning (CCG 2.1, 2.2, 2.3, 2.15, 2.16, 2.20, 2.21). Feedback to 
students should provide specific information related to the goals. 



CLASSROOM LEVEL SPEAKING ASSESSMENT 

Guidelines for Insufficient/Acceptable/Ideal Assessment Practices 



The following examples are intenCid to be illustrative of the procedures and practices 
districts might follow in assessing students' speaking skills. Note that at the first 
(INSUFFICIENT) level, the practices followed, while not necessarily inherently wrong, are 
insufficient to ensure compliance with Standard 602. At the second (ACCEPTABLE) 
level, the practices extend beyond what is described as INSUFFICIENT, and — though not 
ideal — are likely to ensure minimal compliance with Standard 602. At the third (IDEAL) 
level, the practices described are likely to exceed the minimal requirements for 
compliance but still to be within the reach of districts. It is hoped that this IDEAL level 
will serve as a goal for which most districts will aim in practice. 



INSUFFICIENT Teachers primarily assess students' speaking skills through objective 
tests and short response items, with one or two oral reports per year. 
Students may not be aware of the criteria used to judge their oral 
presentations and typically receive a grade after some time has passed 
since the presentation. Feedback to students and parents is in the form 
of a grade which is reported to parents at the end of the term. 
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Students typically do not receive feedback on everyday speaking 
activities, such as class and group discussions, and everyday 
applications. Teacher feedback centers on correctness of content. 

ACCEPTABLE Teachers may administer an objective usage test, but primarily assess 
speaking and speaking-related skills through observation of 
performance. Students will be expected to do individual presentations, 
engage in small and large group discussion, and apply speaking skills in 
everyday situations. Feedback is based on agreed-upon written 
performance criteria which align with the Common Curriculum Goals 
and which are carefully discussed with students in advance of any 
assessment. 

Students have at least three or four opportunities during the school 
year to plan and giv j oral presentations to the class using a variety of 
modes: e.g., recounting of ar experience, persuasive speech, 
intcT^retation of poetry. Criteria for judging performance are tied to 
the specific mode, context or purf)ose of the presentation. 

Assessment incorporates not only evaluation of formal presentations, 
but also observation of students* performance in everyday speaking 
contexts (e.g., during large- and small-group discussions). Students 
hav3 opportunity to provide extended responses during discussions, so 
that assessment can focus not only on the content, but also on the 
student's ability to listen to others, to integrate ideas, to formulate 
new hypotheses and questions, and to establish connections among 
various sources. Other real-life applications might include making a 
phone call or participating in a job interview. Teachers record results 
of students' performance and discuss evaluations with students 
individually in light of agreed-upon criteria and achievement of the 
Common Curriculum Goals. 

IDEAL In addition to the acceptable level, students are expected to make 

more oral presentations and the teacher provides both oral and written 
student feedback. Students have an opportunity to participate in the 
development of performance criteria. 

Students are taught to use the performance criteria. The teacher 
models acceptable and desirable performance to clarify the criteria 
and to give students an opportunity to apply the criteria. 

The teaching and assessment of speaking skills are integrated with 
reading, listening, writing and other curriculum areas. Peer groups are 
used to plan presentations and assess one another's performance. 

Teachers record results of students' performance and track changes 
over time. 




STUDENT ASSESSMENT FOR PROGRAM EVALUATION 

Guidelines for insuff icient/Acceptable/Ideal Assessment Practices 

When evaluating a program, one of the sources of information is student achievement 
data. In addiiion, the program philosophy, goals, materials and other characteristics 
should be reviewed and evaluated. The student achievement data will help. to identify 
where strengths and weaknesses might exist in the current program. The following 
examples are intended to be illustrative of the procedures and practices districts might 
follow in assessing student achievement for program evaluation. At the first 
(INSUFFICIENT) level, the practices followed, while not neressarily inherently wrong, are 
insufficient to meet the standards. At the second (ACCEPTABLE) level, the practices 
would ensure at least minimal compliance with the standards. The third level (IDEAL) 
exceeds minimum compliance but should still be in leach of districts. 



INSUFFICIENT Performance assessment is nonsystematic and is not tied to 

agreed-upon, written criteria. Speaking performance may be assessed 
only as part of a larger high school competency assessment or other 
regular assessment cycle. 

^(♦^(♦♦♦♦♦♦^ 

ACCEPTABLE Students are assessed through a common scoring system based on 

established criteria used in the classroom. Performance is assessed by 
teachers who are trained to use the approved scoring scale. Samples of 
students at several grade levels are assessed periodically. The 
district-level assessment may be based on data collected from 
classroom assessments, provided the district has a means of ensuring 
enough consistency to allow interpretation of that data across 
classrooms. 

IDEAL The link with classroom assessment is particularly strong, with the 

same assessment criteria being used for classroom assessment and 
instruction as is used for program assessment. Classroom teachers, 
trained to use the scoring criteria, assess students* performance in a 
series of common speaking exercises used across classrooms. The 
assessment also incorporates the judgments of other trained teachers 
(e.g., from another classroom, school or district) to help validate the 
scoring. Assessment occurs at least two or three times prior to 
program evalution. 
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APPENDIX A 
SPEAKING ASSESSMENT RESOURCE 



SPEAKING ASSESSMENT RESOURCES 



The following organizations have conducted speaking assessments or have materials 
related to speaking assessment and can provide telephone consultations and limited 
written material. 



Judy Arter 

Northwest Regional Educational Laboratory Test CeuLer 
101 SW Main, Suite 500 
Portland, OR 97208 
275-9562 

The Test Center can provide, bibliographies and sample instruments for checkout and 
review. 



Lc nee Ccoley 
McKay High School 
2440 Lancaster Dr NE 
Salem, OR 97305-1292 
399-3080 

Lance provides training in speaking assessment, especially at the classroom level. 



Robert Martin 

Western Oregon State College 
Monmouth, OR 97361-1394 
838-1220 

Bob provides tra .ing in speaking assessment, especially at the classroom level. 



Rick Stiggins 

Northwesl Regional Educational Laboratory 
101 SE Main, Suite 500 
Portland, OR 97208 
275-9500 

Rick can provide training on applying performance assessment techniques to 
assessing speaking skills in the classroom. 



Salem-Keizer School District 
Charleen Hurst (399-3031) 



Valley Education Consortium 
Glen Fielding (838-1220 x 391) 



INDEPENDENT CONSULTANTS 



Other Speaking Material Available from 
Department of Education 



1. Salem-Keizer School District. Training videotape and support materials to train 
teachers to rate student speaking skills. (Available for checkout and review from the 
Department of Education Information Resource Center, 378-8471. Materials may be 
purchased for $9.50 plus shipping from Video Transfer Center, 1501 SW Jefferson, 
Portland, 97201, 226-5091.) 



2. Valley Education Consortium. Procedures for Assessing Selected Speaking and 
Listening Skills . 1984. 

Provides sample procedures to assess student speaking skills in grades K-12. Includes 
non-verbal communication, performing social rituals, giving directions and 
summarizing messages orally, expressing and supporting an idea or point of view, and 
effective use of voice. It also includes an analytic scale for evaluating speaking. 



3. Gere, Anne Ruggles and Eugene Smith. Attitudes, Language and Change . Urbana, 
IL: NCTE (Stock No. 02174), 1979. 

Includes several self-assessment checklists on appropriate language usage for formal 
or informal situations. (Available for check-out and review from the Department of 
Education, Information Resource Center, 378-8471. It may be purchased for $7.50 
plus $1.25 shipping from NCTE, 1111 Kenyon Drive, Urbana, IL 61801.) 

4. D evelopment of an Oral Communication Assessment Program: The Glvnn Countv 
Speech Proficiencv Examination for High School Students . Glynn County School 
System, Brunswick, GA, 1981. 

This report on the development of a speaking performance assessment includes an 
interview and a public hearing testimony simulation. Available for check-out and 
review from the Department of Education, Information Resource Center, 378-8471. 



5. Listeninp. and Speaking in the English Language Arts Curriculum K-12: A Manual for 
Teachers . New York State Department of Education, 1988 (Draft). 

Provides a variety of activities at all grade levels including several self-evaluation 
forms and teacher checklists. Available for check-out and review from the 
Department of Education, Information Resource Center, 378-8471. 
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APPENDIX B 

SPEECH COMMUNICATION ASSOCIATION REFERENCES 
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CRITERIA FOR EVALUATING INSTRUMENTS AND PROCFOURES 

FOR ASSESSING SPEAKING AND LISTENING* 

The following criteria may be applied to published and unpublished instruments and 
procedures for assessing speaking and listening skills of children and adults. The criteria 
are organized around (a) content considerations, which deal primarily with the substance 
of speaking and listening instruments and procedures, and (b) technical considerations, 
which deal with such matters as reliability, validity and information on administration. 

1. Stimulus materials should require the individual being tested to demonstrate 
skill as a speaker or listener, 

2. Assessment instruments and procedures should clearly distinguish speaking and 
listening performance from reading and writing ability; i.e., inferences of 
speaking and listening competence should not be made from tests of reading 
and writing, and directions and responses for speaking and/or listening tests 
should not be mediated through reading and writing modes. 

3. Assessment instruments and procedures should be free of sexual, cultural, 
racial, and ethnic content and/or stereotyping. 

4. Assessment should confirm the presence or absence of skills, not diagnose 
reasons why individuals demonstrate or fail to demonstrate those skills. 

5. Assessment should emphasize the application of speaking and listening skills 
that relate to familiar situations; i e., stimulus materials should refer to 
situations recognizable to the individi il being tested and should facilitate 
demonstration of skills rather than demonstration of content mastery. 

6. Assessment should test skills that are important for various communication 
settings (e.g., interpersonal small group, public, and mass communication 
settings) rather than be limited to one setting. 

7. Assessment should permit a range of acceptable responses, where such a range 
is appropriate. 

8. Assessment should demonstrate that outcomes are more than just chance 
evidence; i.e., assessment should be reliable. 

9. Assessment should provide results that are consistent with other evidence that 
might be available. 

10. Assessment should have content validity. 

11. Assessment procedures should be standardized and detailed enough so that 
individual responses will not be affected by the administrator's skills in 
administering the procedures. 

12. Assessment procedures should approximate the recognized stress level of oral 
communication; they should not increase or eliminate it. 

13. Assessment procedures should be practical in terms of cost and time. 



Reprinted with permission of Speech Communication Association. 
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14. Assessment should involve simple equipment. 

15. Assessment should be suitable for the developmental level of the individual 
being tested. 



♦Developed by Philip M. Backlund, Kenneth L. Brown, Joanne Gurry, and Fred E. Jandt 
acting as a subgroup of the Speech Communication Association's Educational Policies 
Board Task Force on Assessment and Testing. Approved and endorsed by the Educational 
Policies Board and the Administrative Committee of the Speech Communication 
Association. 
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SPEECH COMMUNICATION ASSOCIATION 
COMMITTEE ON ASSESSMENT AND TESTING 
SUBCOMMITTEE PUBLICATIONS AND PAPERS 
1979 - 1984 



Backlund, P. (1981). A National Survey of State Practices in Speaking and Listening 
Assessment. In R. Stiggins (Ed.), Perspectives on the Assessment of Speaking and 
Listening Skills for the 1980s . Portland, OR: Northwest Regional Educational 
Laboratory. 

Backlund, P.M., Brown, K.L., Gurry, J., & Jandt, F. (1979). Criteria for Evaluating 

Instruments and Procedures for Assessing Speaking and Listening. SPECTRA . 15, 5. 
Also published in the 1982 Resources for Assessment in Communication . Annandale, 
VA: Speech Communication Association. 

Backlund, P.M., Brov/n, K.L., Gurry, J., & Jandt, F. (1980). Evaluating Speaking and 
Listening Skill Assessment Instruments: Which one is Best for You? Language Arts, 
57, 621-627. 

Backlund, P.M., Brown, K.L., Gurry, J., & Jandt, F. (1982). Recommendations for 
Assessing Speaking and Listening Skills. Communication Education, 3K 9-18. 

Backlund, P., Booth, J., Moore, M., Parks, A. M., & Van Rheenen, D. (1982). A National 
Survey of State Practices in Speaking and Listening Skill Assessment. 
Communication Education, 3K 125-130. 

Brown, K. L. (1979, October). Assessment of basic oral communication skills: A selected 
annotated bibliography. Annandale, VA: ERIC and Speech Communication 
Association. 

Brown, K. L., Backlund, P., Gurry, J., & Jandt, F. (1979). As sessment of Basic Speaking 
and Listening Skills: State of the Art and Recommendations for Instrument 
Development: Vols. 1 and 2. Boston: Bureau of Research and Assessment, 
Massachusetts Department of Education. 

Gray, P. A. (1984). Assessment of basic oral communication skills: A selected, annotated 
bibliography. Annandale, VA: ERIC and Speech Communication Association. 

McCaleb, J. (1983, February). An analysis of measures of teachers' oral 

communications. Paper presented at the meeting of the American Association of 
Colleges for Teacher Education, Detroit. 

McCaleb, J. (1983). The assessment of oral communications of teachers. Teacher Talk, 
2, 5-6. 

Moore, M. (1984, July). State and higher education efforts in listening assessment: A 
status report. Paper presented at the 1st annual summer conference of the 
International Listening Association, St. Paul, MN. 
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Moore, M. (1984). Current state department listening efforts. In C. Coakley (Ed.), 
Listening: Competer cies, curriculum and assessment challenges confronting state 
departments of edu( ation (Proceedings of the 1st Annual Summer Conference of the 
International Listering Association). St. Paul: International Listening Association. 

Rubin, D. L. (1981). Usirg performance rating scales in large scale assessment of speech 
proficiency. In R. itiggins (Ed.), Perspectives on Oral Communication Assessment 
for the 1980*s. Portland, OR: Northwest Regional Educational Laboratory Center for 
Applied Performan:-^ Testing. 

Rubin, D. L. (1981, April). Uses of Pertormance Rating Scales in Large Scale Assessment 
of Oral Communication Proficiency. Paper presented at the meeting of the 
American Educational Research Association, Los Angeles. 

Rubin, D. L. (1981, November). Review of Speaking and Listening Tests, Prek - 12. 

Paper presented at the meeting of the Speech Communication Association, Anaheim, 
CA. 

Rubin, D. L. (1981, November). Evaluating Communication Attitudes. Paper presented at 
the meeting of the National Council of Teachers of English, Boston. 

Rubin, D. L. (1982). Review of Brendon J. Carroll, Testing Communicative Performance: 
An Interim Study. English Education, 14, 99-101. 

Rubin, D. L. (1983). Testimony submitted to the National Commission on Excellence in 
Education, United States Department of Education, submitted by the Speech 
communication Association Committee on Assessment and Testing Communication 

) Education, 32 439-441. 

Rubin, D. L. (1983, November). Developments in teaching, assessing and disseminating 
oral communication, K-College. Paper presented at the meeting of the Speech 
Communication Association, Washington, DC. 

Rubin, D. L., Daly, J. A., McCroskey, J.C., & Mead, N. A. (1982). A review and critique 
of procedures for assessing speaking and listening skills among Kindergarten through 
grade 12 students. Communication Education, 31 , 285-303 

Rubin, D. L., Mead, N. A. (1984). Large-scale assessment of oral communication skills: 
Kindergarten through grade 12. Annandale, VA: ERIC/RCS and the Speech 
Communication Association. 

Rubin, R. B. (1982, October). Assessing college communication competencies. Paper 
presented at the meeting of the Speech Communication Association of Ohio, 
Columbus, OH. 

Rubin, R. B. (1982, November). Oral communication assessment procedures and 

instrument development in higher education. Paper presented at the meeting of the 
Speech Communication Association, Louisville, KY. 

Rubin, R. B. (1984). Communication assessment instruments and procedures in higher 
education. Communication Education, 33, 178-180. 

Rubin, R. B. Moore, M. R., Sisco, J. I. & Quianthy, R. (1983). Oral communication 

assessment procedures and instrument development in higher education. Annandale, 
VA: Speech Communication Association. 
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ASSESSMENT OF BASIC ORAL COMMUNICATION SKILLS 



A Selected, Annotated Bibliography 



Prepared by 



Philip A. Gray 
Department of Communication Studies 
Northern Illinois University 
May 1984 



Distributed by the Speech Communication Module-ERIC Clearinghouse on Reading 
and Communication Skills, 5105 Backlick Road, Annandale, VA 22003. This 
bibliography may be reproduced for free distribution without permission of the 
Speech Communication Module. Ordering information for ERIC materials (ED and 
EJ numbers) can be found on page 4 of this bibliography. 



This bibliography includes materials for educators who are concerned with assessment of 
basic speaking and listening skills, especially in the context of minimal competency 
testing and basic skills improvement programs. The bibliography is divided into two 
sections. The first includes materials that address broad assessment issues, review a 
variety of test instruments, and report assessment practices throughout the states. The 
second contains sources that focus specifically on the assessment of speaking, listening, 
and functional communication skills. 



GENERAL SOURCES ON ASSESSMENT ISSUES AND INSTRUMENTS 



Achievement Testing and Basic Skills. Proceedings of the national conference on 
achievement testing and basic skills. Washington, D.C.: U.S. Department of Health, 
Education and Welfare and National Institute of education, 1979. This report covers 
various basic skills i*:sues — use of test results, implications of tests for poor and 
minority children, cultural considerations, and the Federal role in testing. 

Bloom, B.S.; Hastings, J.T.; and Maddaus, G.F. Handbook on Format i ve and Summative 
Evaluation of Student Learning. New York: McGraw-Hill Book Co., 1971. 
Distinguishes between formative and summative evaluation with chapters on 
evaluating language development in preschool education and elementary school 
language arts. Each chapter presents objectives, illustrates testing procedures, and 
discusses commercial tests. 




Reprinted by permission of the Speech Communication Association 
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Bostrom, R., (ed.) Competence In Communication. An Interdisciplinary Perspective . 
Beverly Hills, CA: Sage Publications, 1984. Book examines crosscultural, 
interpersonal, organizational, developmental, and mass communication perspectives 
on competence in communication. 

Buros Institute of Mental Measurements, James V. Mitchell, Jr., (Ed.). Tests In Print III . 
Lincoln, Nebraska: University of Nebraska Press, 1983. A standard reference for 
information on published tests. Lists an index to tests, test reviews, and literature 
on specific tests. 

Clark, J. P., and Thomson, S.D. Competency Tests and Graduation Requirements . 
Reston, VA: National Association of Secondary School Principals, 1976. (ERIC ED 
126 160; available in microfiche from EDRS.). Provides a background to the 
competency testing movement and reports on nationwide initiatives in the use of 
applied performance tests. Reviews a variety of competency tests that measure 
skills achievement. The revised edition by James W. Keefe and Nancy De Leonibus is 
available in microfiche from EDRS (ERIC ED 194 573); paper copy can be purchased 
from NASSP, 1904 Association Dr., Reston, VA 22091. 

Dickson, W.P., (Ed.) Children's Oral Communication Skills . New York, Academic Press, 
1981. Chapters on a process-structuralist view of communication competency, 
cognitive and comprehension monitoring, skill acquisition, etc. 

Pagan, W.; Cooper, C; and Jensen, J. Measures for Research and Evaluation In the 
English Language Arts . Urbana, II: National Council of Teachers of English, 1975. 
(ERIC ED 099 835). Reviews over 100 unpublished instruments for assessing language 
development, listening, and standard English as a second language, as well as 
literature, reading, teacher competency, and miscellaneous language skills. 

Fisher, B.A., (Ed.). Western Journal of Speech Communication . Western Speech 
Communication Association (Lynn Wells, Executive Secretary, Saddleback 
Community College). (Vol. 48, No. 2, Spring 1984). A special issue on children's 
communicative development including conversational competency, acquiring 
sociclinguistic knowledge, differences between comprehension and production of 
language, impact of TV advertising, role-taking, and development from birth. 
(Indexed in ERIC CUE ). 

Haney, W. Standards for Tests and Test Use , Staff Circular No. 3. Cambridge, MA: 
Huron Institute, 1978. Reviews six sets of standards concerning standardized tests 
(including those of APA and AERA) and discusses the impact of these standards on 
testing practices. 

Haney, W. Testing the Tests . Staff Circular No. 1. Cambridge, MA: Huron Institute, 
1978. Reviews past efforts to rate the quality of standardized tests. Discusses 
implications as well as problems in assessment of test quality. 

Johnson, O.G. Tests and Measurement In Child Development: Handbook 11 . (Vols 1 and 
2). San Francisco: Jossey-Bass Publishers, 1976. Describes 900 unpublished tests and 
measures of child development (birth through age 18). The measures cire classified in 
18 major categories. 
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Perspectives on communication competency. Three articles offering differing 
perspectives on competency. See James C. McCroskey, "Ccnimunication 
competence and performance: a research and pedagogical perspective," 
Communication Education , (Vol. 31, No. 1, Jan. 1982, pp 1-7). McCroskey argues the 
need to separate concepts of communication competency anc communication 
performance. (ERIC ED 203 401). Gerald M. Phillips, "A competent view of 
'competence'." Communication Education , (Vol. 33, No. 1, Jan 1984, pp 25-36). 
Phillips addresses the preoccupation with definitions and measurement of 
competence. He suggests that the locus of interest should be shifted to techniques 
for traiPing performance improvement. And, Brian H. Spitzberg, "Communication 
competence as knowledge, skill, and impression," Communication Education , (Vol. 32, 
2, July 1983, pp 323-329). This essay proposes that competence be viewed as a 
function of knowledge, skill, and motivation. Compares this perspective with 
McCroskey's. (Last two articles indexed in ERIC CUE .). 

Perspectives on the Assessment of Speaking and Listening Skills for the 1980's . AERA 
Symposium. Published by Northwest Regional Educational Laboratory, 1981. (ERIC 
ED 210 748; also available from SCA). 

Petty, W.T. and Fielding, L. Developing Children's Language . Boston: Allyn and Bacon, 
1980. Defines language and language arts, surveys language learning, and describes 
the development of language instruction. Several chapters on listening and oral 
language and expression. 

Rubin, R.B., "Communication assessment instruments and procedures in higher 
education," Communication Education , (Vol. 33, No.2, April 1984, pp 178-180). A 
summary report by the SCA Committee on Assessment and Testing describing the 
range and degree of assessment occurring in colleges and universities. (Indexed in 
ERIC CUE). 

Scott, L. (Ed.). Summary of the Fall 1978 Conference of the National Consortium on 
Testing . Cambridge, MA: Huron Institute, 1978. Describes current state of 
criterion-referenced testing, alternative approaches to assessment and reviews of 
major standardized test series. 

Simon, A., and Boyer, E.G. Mirrors for Behavior III: An Anthology of Observation 
Instruments . Wyncote, PA: Communication Materials Center, 1974. A review of 
observation systems that measure various dimensions of the communication process 
in the classroom. 

State Practices in Speaking and Listening Assessment . Speech Communication 
Association (No. 423). Article and backup information on state positions for K-12 
programs in speech. 

Wiemann, J.M., and Backlund, P.M., "Current theory and research in communicative 
competence," Review of Educational Research , (Vol. XV, Spring 1980; pp 185-199). 
Includes a review of relevant research. (Also see ERIC ED 155 763.). 
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SOURCES ON ASSESSMENT OF SPEAKING, LISTENING, 
AND FUNCTIONAL COMMUNICATION SKILLS 



Allen, R.R., and Brown, K.L. (Eds.). Developing Communication Competence in 
Children . Skokie, IL: National Textbook Co., 1976. (May be purchased from SCA). 
Literature review, behavioral study of children's communication competencies, and 
an assessment of teachers* attitudes regarding goals of speech communication 
instruction. Also contains a framework and objectives for functional communication 
instruction. 

Allen, R.R., and Wood, B.S., "Beyond reading and writing in communication competence," 
Communication Education . (Vol. 27, Nov. 1978, pp 286-292). Argues for developing 
competencies in an array of communication situations involving speaking and 
listening as well as reading and writing. Five functions of communication are 
offered as the communication focus of a language arts program. (ERIC EJ 201 016). 

Backlund, P.; Gurry, J.; Brown, K.; and Jandt, F., "Evaluating speaking and listening skills 
assessment instruments: Which one is best for you?" Language Arts , (Vol. 57, No. 6, 
Sept. 1980, pp 621-627). Focuses on need for assessment of skills that accounts for 
differences between those unique to oral language and those unique to written 
language. (ERIC EJ 233 979). 

Backlund, P.; Gurry, J.; Brown, K.; and Jandt, F., "Recommendations for assessing 
speaking and listening skills," Communication Education . (Vol. 31, No. 1, Jan. 1982, 
pp 9-18). Authors continue to clarify and develop criteria introduced in the 1980 
article. (ERIC EJ 257 625). 

Bassett, R.E.; Whittington, N.; and Staton-Spicer, A., "The basics in speaking and 
listening for high school graduates: What should be assessed?" Communication 
Education . (Vol. 27, Nov. 1978, pp 293-303). Recommends twenty speaking and 
listening competencies for high school graduates. Illustrates how each competency 
can be applied in occupational, citizenship, and maintenance situations. (ERIC EJ 
201 017). 

Faires, C.L., "The development of listening tests," (ERIC ED 220 528). Paper presented 
at the annual meeting of the Mid-South Education Research Association, Nov. 1980. 
Critical of most test instruments for poor construction, sample size, reporting of 
test item analysis data, and lack of replication. 

Illinois State Board of Education, Assessing Oral Communication Skills , Springfield, IL: 
ISBE, (free from ISBE, 100 North First Street, Springfield, II, 62777). Booklet 
suggesting classroom observation techniqi.es for assessing oral communication skills. 
Uses a checklist approach 

Larson, C.E., "Problems in assessing functional communication," Communication 
Education. (Vol. 27, Nov 1978, pp 304-309). Emphasizes that problems in assessing 
functional communication have their origin in conceptual ambiguity. Contrasts 
communicative competence with functional effectiveness and proposes the latter as 
a more appropriate construct for older children and adults. (ERIC EJ 201 018). 
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Larson, C; Backlund, P; Redmond, M.; and Barbour, A. Assessing Functional 

Communication . Falls Church, VA; Speech Communication Association, 1978. (ERIC 
ED 153 275; also available from SCA). Part I identifies and describes conceptual and 
methodological issues involved in evaluating the major components of interpersonal 
interaction related to functional communication. Part 11 contains brief reviews of 90 
instruments designed to generate information on some aspect of functional 
communication. 

Lederman, L.C., and Ruben, B.D., "Systematic assessment of communication games and 
simulations: an applied framework," Communication Education , (Vol. 33, No. 2, 
April 1984, pp 152-159). Establishes a framework for the assessment of 
communication games and simulations. Specifies criteria and provides a model for 
the selection/design, use, and assessment of activities. (Also indexed in ERIC CUE). 

Lundsteen, S.W. Listening: Its Impact At All Levels On Reading And The Other Language 
Arts . (Rev. ed.). Urbana, II: National Council of Teachers of English, 1979. (ERIC 
ED 169 537). A review of research in listening including definitions and evaluation 
methods. Presents a listening taxonomy and describes commercial, unpublished, and 
teacher-designed assessment instruments and procedures. 

Monge, P.R.; Bachman, S.G.; Dillard, J. P.; and Eisenberg, E.M., "Communicator 
competence in the workplace: Model testing and scale development," 
Communication Yearbook 5 . (M. Burgoon, ed.) New Brunswick, NJ: Transaction 
Books, 1981, pp 505-527. Assessing competencies on the job. 

Rubin, D.; Daly, J.; McCroskey, J.; and Mead, N., "A review and critique of procedures 
for assessing speaking and listening skills among preschool through grade twelve 
students," Communication Education . (Vol. 31, No. 4, Oct. 1982, pp 285-304). 
Review and critique of 45 available listening and speaking assessment instruments. 
Recommendations for further research and development. (ERIC EJ 269 962). 

Rubin, R., "Assessing speaking and listening competence at the college level: the 
communication competency assessment instrument," Communication Education . 
(Vol. 31, No. 1, Jan. 1982, pp 19-32). Report on the CCAI, a test of basic 
communication skills for college students. (ERIC EJ 257 626). 

Rubin, R.B. Communication Competency Assessment Instrument , 3/4" video stimulus 
tape (No. 901, $85), testing manual (No. 902, $15), 50 rating sheets (No. 903, $9.50). 
Also available as a starter package for $198 (1 tape, 3 manuals, 10 pkgs. rating 
sheets) (No. 900) from SCA. A college level assessment instrument designed to 
measure 19 speaking and listening competencies. 

Rubin, R.B.; Moore, M.R.; Sisco, J.; and Quianthy, R. Oral Communication Assessment 
Procedures and Instrument Development in Higher Education (No. 426) (ERIC ED 236 
721; also available from the Speech Communication Association.) Report by the SCA 
Subcommittee for Oral Communication Assessment in Higher Education. 

SCA Guidelines for Competencies in Speaking and Listening for High School Graduates . 
Brochure available from the Speech Communication Association. Single copy free 
with self-addressed stamped envelope. (No. 414). 
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Stohl, C, "Developing a communicative competence scale," Communication Yearbook 1 . 
(Robert N. Bostrom, ed.) Beverly Hills: Sage Publications, 1983, pp 685-716. Study 
defines communicative competence (focusing on 3-5 year old children), briefly 
reviews methods of assessment, and provides a report of the empirical investigation. 

Trank, D.M., and Steele, J.M,, "Measurable effects of a communication skills course: an 
initial study," Communication Education . (Vol. 32, No. 2, April 1983, pp 227-236). 
Study examined the amount of growth in both writing and peaking competence over a 
semester of instruction. Utilized the speech and writing portions of ACT's College 
Outcome Measures Project. (ERIC EJ 277 822). 

Wood, B.S. (Ed.). Development of Functional Communication Competencies: Pre-K ~ 
Grade 6. and Development of Functional Communication Competencies: Grades 
7-12 . Falls Church, VA: Speech Communication Association, 1977. (ERIC ED 137 
858 and 859; also available from SCA.). The first describes the child as a competent 
communicator and discusses techniques that encourage functicnal communication 
competence. The second booklet focuses upon the adolescent and teenager. 

Work, W., "ERIC report: testing - who's being tested, by whom, by what means, for what, 
and how well?" Communication Education . (Vol. 216, 1977, pp 338-344). Provides a 
sample of ERIC documents representative of those in the system focusing on the 
evaluation of communication instruction programs. (ERIC EJ 173 268). 



ED numbers refer to education documents in Resources in Education (RIE), the ERIC 
monthly abstract journal. Unless otherwise indicated, documents are available on 
microfiche in libraries housing ERIC collections, or they can be purchased in microfiche 
or reproduced paper copy from ERIC Document Reproduction Service (EDRS), Box 190, 
Arlington, VA 22210. For prices check latest issue of RIE or write to the Speech 
Communication Module for an EDRS order blank/price form. Several of the ERIC sources 
can be obtained in published paper book directly from the Speech Communication 
Association (SCA), 5150 Backlick Road, Suite E, Annandale, VA 22003. 

EJ numbers are education journal articles indexed in Current Index to Journals in 
Education (CUE), the ERIC monthly index. Reprints from these articles are available 
from University Microfilms International. Current issues of Communication Education 
can also be purchased from the Speech Communication Association; back issues from the 
Johnson Co., 355 Chestnut St., Norwood, NJ 07468. 
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APPENDIX C 



SPEAKING ASSESSMENT PROCEDURES 
MASSACHUSETTS DEPARTMENT OF EDUCATION 



THE SPEAKING ASSESSMENT 



Objectives and Assessment Design 

The speaking assessment was designed to measure the 14 speaking objectives identified by 
the Massachusetts Basic Skills Improvement Policy (see Exhibit 2). Like the listening 
objectives, some of the speaking objectives deal with general speaking skills that apply to 
all speaking situations, for example. Objective A-1, Use Words and Phrases Appropriate 
to the Situation. Other objectives deal with specific speaking situations, for example. 
Objective C-1, Use Survival Words To Cope With Emergency Situations. 

A set of speaking rating scales were developed for the speaking assessment. These scales 
were used in two ways: 

• For classroom observation by teachers. 

• For one-on-one assessment by trained raters. 

The following sections describe the speaking rating scales, the teacher observation 
approach, the one-on-one approach and how the approaches are combined into a 
two-staged assessment. 

Speaking Rating Scales 

A set of four scales was developed to measure the objectives that deal with general 
speaking skills. Each scale measures one dimension of speaking skills. Thp dimensions 
are: 

• Delivery 

• Organization 

• Content 

• Language 

The delivery dimension focuses on Objectives A-2, A-3 and A-4 and is concerned with 
how well a student transmits messages. It assesses how well a student uses appropriate 
volume, rate and articulation while speaking. The organization dimension focuses on 
Objectives B-1 and B-2 and is concerned with how well a student structures messages. It 
assesses how well a student expresses the sequence or the relationship of ideas. The 
content dimension focuses on Objectives A-1, B-3, B-4 and C-2 and is concerned with 
how well a student provides an adequate amount of relevant information to meet the 
requirements of various speaking tasks. In addition, it assesses how well a student adapts 
the content of messages to specific listeners and situations. The language rating focuses 
on Objective B-5 and is concerned with how well a student uses appropriate grammar and 
vocabulary while speaking. 
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Within each speaking dimension, performance is rated using a 4-point scale: 

1 = Inadequate 

2 = Niinimal 

3 = Adequate 

4 = Superior 

The dimensions and the levels of the rating scales are explained in the "Speaking 
Assessment Ratings Guide," found in Appendix B. 



The Teacher Observation Approach 

The teacher observation approach is a general measure of a student's speaking 
performance. In the statewide assessment, two teachers (who had the same student 
currently enrolled in class) independently rated the student's general speaking 
performance in class using the speaking rating scales. Usually one of the teachers was an 
English teacher and the other from another subject area where students participate in a 
fair amount of classroom discussion and interaction, such as history, government or 
science. 

Teachers were asked to read the "Speaking Assessment Ratings Guide" and then to 
complete the ratings. Teachers based their ratings of a student on their observation of 
the student's performance in normal classroom activities such as asking questions, 
responding to questions, explaining how to do something, giving a report to the class, or 
talking with other students in discussion groups. They considered the student's overall 
performance since the beginning of the semester. 



The One-on~One Approach 

The one-on~one approach is a focused measure of a student's speaking performance. In 
the statewide assessment, a trained rater rated the student's performance using the same 
speaking rating scales used by teachers for the teacher observation approach. However, 
instead of basing ratings on classroom observations, the rater gave the student specific 
speaking tasks and rated the student's oral response to these tasks. 

Each rater was provided with one day of training in the one-on-one approach. The rater 
assessed each student individually. The rater gave the student several speaking tasks and 
rated the student's performance on ea.h task along the four dimensions. Thus, for each 
task, the rater gave the student a rating from 1 to 4 for delivery, organization, content 
and language. The ratings were assigned immediately after the student's response. 

The speaking tasks used in the one-on~one approach reflect the objectives that deal with 
specific speaking situations. The tasks include: 

• A description 

• An emergency 

• A sequence 

• A persuasion task 
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The description task focuses on Objective C-4 and is concerned with how well a student 
can describe an object, event or experience so that another person would know something 
about the topic. The emergency task focuses on Objective C-1 and is concerned with how 
well a student can provide the necessary information in an emergency so that another 
person could send help. The sequence task focuses on Objective and is concerned 
with how well a student can explain a sequence of steps so that another person could 
follow the sequence. The persuasion task focuses on Objective C-5 and is concerned with 
how well a student can present effective arguments so that another person would be 
persuaded by the student's point of view. 

The focus of all of the speaking tasks was on the student's effectiveness in transmitting 
messages — not on the specific content of the messages. Tasks were developed that were 
familiar to all students and did not require any special knowledge or experience. The 
tasks were field tested with 9th graders and 12th graders from inner-city, suburban and 
rural schools to assure that they would be relevant for a wide variety of students of 
various ages. 

Based on field test results, four tasks (one of each type) were selected for the final 
version of the one-on-one speaking test used for the statewide assessment and four 
additional tasks for each of the three alternate forms of the test. The tasks were 
selected so that they were as similar as possible in content. These tasks were approved 
by the Department of Education's Bureau of Equal Educational Opportunity as being 
"substantially free of offensive sexual, cultural, racial and ethnic content and 
stereotyping." Further information from the statewide assessment was used to create 
three alternate forms of the test that were equivalent in difficulty. The text of the state 
speaking tasks is provided in Appendix C. 



The Two-Staged Approach 

The rationale for developing a two-staged approach for assessing speaking is based on the 
need for measures that are reliable, valid and free of bias. The teacher observation 
approach provides a general measure of student performance. It assesses all types of 
speaking tasks as they occur in a natural situation. However, sometimes a general 
measure such as this one allows for extraneous factors, such as academic achievement or 
sociability, to enter into the ratings. The one-on-one approach provides a focused 
measure of student performance. It assesses only a few types of speaking tasks in a 
contriving setting. However, it focuses entirely on speaking variables and uses 
standardized procedures. The two approaches complement one another and, taken 
together guard against the many problems of reliability, validity and bias that are a 
particular concern in speaking measures. 

Another reason for the two-staged approach is the need for measures that are feasible 
for large-scale use by school districts in the future. The intention is to use the teacher 
observation approach as a screening measure and to use the one-on-one approach as a 
back-up measure in cases where a student's level of ability is in question. School 
districts would use the teacher observation measure to assess all students in a grade. 
They would use the one-on-one measure to assess students whose level of ability is in 
question. The one-on-one measure would be used when two teachers do not agree in their 
observation ratings of a student. 



Before final plans are made for implementing the speaking measures in school districts, it 
is necessary to demonstrate that the two approaches are reliable and valid. Initial tests 
of the reliability and validity of the measures were conducted during the statewide 
assessment. However, further study will be undertaken before final recommendations are 
made regarding how the measures should be used by school districts. The instrument is 
still in the developmental stages and has not yet been approved for use in implemental 
stages and has not yet been approved for use in implementing the speaking component of 
the Basic Skills Improvement Policy . 



Results of the Speaking Assessment 

On the average, students performed adequately in all areas of the speaking assessment. 
Their average ratings for the teacher observation approach and the one-on-one approach 
were about the same, both falling just below 3 on a 4-point scale. 

The average rating of students for the teacher observation ratings across all dimensions 
was 2.95. The averages for each speaking dimension ranged from 2.91 for Delivery to 
2.99 for Content. Thus, the level of performance did not vary much from dimension to 
dimension. 

The average rating of students on the one-on-one ratings across all dimensions and tasks 
was 2.89, slightly lower than the average rating for the teacher observation ratings. The 
ratings summarized across speaking dimensions ranged form 2.85 for Content to 2.94 for 
Language. The ratings summarized across speaking casks ranged from 2.84 for 
Description to 2.97 for Emergency. Thus, similar to the teacher observation results, the 
level of performance on the one-on-one ratings did not vary much from dimension to 
dimension or from task to task. 

Most of the ratings (79 percent) assigned to students in the one-on-one assessment were 
3s which reflect adequate performance. Very few ratings (about 1 percent) were Is, 
inadequate performance and few rating (about 6 percent) were 4s, superior performance. 
Of interest is the small but noticeable percentage of 2 ratings (about 17 percent), which 
reflect minimal performance. Another result that demonstrates a lack of skills among 
some students is the failure of 18 percent of the students to mention the nature and 
location of the emergency in the emergency task. 



Reliability and ValiditVLof the Speaking Approaches 

In addition to the analysis of student performance, several steps were implemented prior 
to and during the assessment to test the reliability and validity of the speaking 
approaches. 

The reliability of the teacher observation approach was determined by examining the 
consistency of the ratings of two teachers of the sa.ne student. Approximately 95 
percent of the ratings were either identical or adjacent (within one point of one another). 
There were no systematic differences in the ratings of different types of teachers; i.e., 
English teachers compared with other teachers. 



The reliability of the one-on-one approach was determined by examining the consistency 
of the ratings of the trained raters for the same student. In a test of inter-rater 
reliability conducted prior to the assessment, raters assigned identical ratings 85 percent 
of the time. Rater consistency was determined after the assessment by rescoring 10 
percent of the students' responses. Rescoring was conducted by project staff who were 
reliable raters. In this test, raters assigned identical ratings 75 percent of the time. 

The validity of the speaking approaches was determined by expert judgment and empirical 
tests. Prior to the assessment, a panel of communication experts independently examined 
and speaking tasks and speaking dimensions and in a blind review, matched them to the 
speaking objectives. In general, the panel agreed almost unanimously with respect to the 
categorization of tasks and most of the time with respect to the categorization of 
speaking dimensions. Based on the reviewers' comments, adjustments were made in the 
tasks and dimensions so that they better reflected the objectives. 

The degree to which the two assessment approaches were measuring the same thing, 
concurrent validity, was determined by comparing the teacher observation ratings and 
one-on-one ratings for the same student. Ratings were combined to form a total teacher 
observation score and an adjusted total one-or-one score, both scores ranging from 8 to 
32. In 81 percent of the cases the scores were within 4 points of one another and in 98 
percent of the cases scores were within 8 points of one another. 



BACKGROUND INFORMATION 



Introduction 

The main purpose of the statewide assessment is to collect information on the listening 
and speaking performance of 12th graders. A secondary purpose of the assessment is to 
collect background information that provides additional insight into the performance of 
students and baseline information prior to the implementation of the Basic Skills 
Improvement Policy . 

Background information was collected through questionnaires administered to principals, 
teachers and students participating in the assessment. Some of the questions focused on 
the nature of the school environment, including general descriptive information and 
specific aspects of school programs that might influence listening and speaking skills. 
Other questions focused on characteristics of the student, including general descriptive 
information and specific factors that might impact listening or speaking skills. 

Some of the background questions asked for factual information. All of the background 
questions rely upon the self-reported responses of principals, teachers and students. 
Thus, the data reflect the perceptions and opinions of the individuals responding to the 
questions. Also, since the sanple was designed to be representative of students, not 
schools, the results of the Principal Questionnaire cannot be generalized to all secondary 
schools in Massachusetts. 



Results of Background Questionnaires 



The results of the Principal Questionnaire indicate that most schools i ^luded some 
listening and speaking skills in the curriculum and extracurricular program. However, 
principals felt there was deficit in the skills of many students and a need for more 
instruction. Sixty-four percent of the principals stated that there was not enough 
emphasis on listening instruction and 69 percent stated that there was net enough 
emphasis on speaking instruction. Fifty-three percent of the principals felt that students 
can graduate without knowing how to listen or speak very well. 

The results of the Student Questionnaire indicate that they do not receive much 
instruction in listening or speaking but that they believe these sVills are important. 
Ninety-three percent of the students have rarely been given work in school aimed at 
improving listening skills and 90 percent have rarely been given work in school aimed at 
improving speaking skills. Eighty-three percent stated that students should learn the 
skills in the listening test and 92 percent stated that students should be able to do the 
tasks in the one-on-one speaking test before they finish high school. However, almost 
half of the students felt that students currently can graduate without knowing how to 
speak or listen very well. 

The results of the Teacher Questionnaire indicated that participating teachers did not 
have major problems implementing the teacher observation ratings. Eighty-six percent 
of the teachers had no trouble understanding the "Speaking Ratings Guide" and over 90 
percent felt confident about the accuracy of the ratings they made of students' speaking 
performance. 



USE OF MASSACHUSETTS BASIC SKILLS TESTS AND MATERIALS 



The listening and speaking tests and accompanying materials are restricted for use in 
Massachusetts schools for the specific purpose of implementing the Massachusetts Basic 
Skills Improvement Policy and Regulations . Circulation, distribution, duplication or use 
for other purposes requires written permission from the Massachusetts Department of 
Education. 

Researchers, Department of Education officials from other states and personnel from 
school districts outside Massachusetts interested in using these materials for data 
collection and/or further research and development are encouraged to contact the 
Massachusetts Department of Education. In general, permission for use will be granted 
for appropriate purposes, subject to certain conditions. 



Attachment A 



MASSACHUSETTS DEPARTMENT OF EDUCATION ASSESSMENT OF BASIC SKILLS 
SPEAKING ASSESSMENT RATINGS GUIDE 



OVERVIEW 



There are numerous kinds of speaking tasks that students must perform in everyday life, 
both in school and out of school. The Massachusetts Basic Skills Improvement Policy has 
focused on some of these tasks, including describing objects, events and experiences, 
explaining the steps in a sequence, providing information in an emergency and persuading 
someone. 

In order to accomplish a speaking task, the speaker must formulate and transmit a 
message to a listener. This procc*ss involves deciding what needs to be said, organizing 
the message, adapting the message to the listener and situation, choosing language to 
convey the message and finally delivering the message. The effectiveness of the speaker 
may be rated in terms of how well the speaker meets the requirements of the task . 

The Massachusetts test of basic skills in speaking separates speaking skills into four 
dimensions: 

Delivery 
Organization 
Content 
Language 

Delivery is concerned with the tr?nsmission of the message, i.e., volume, rate and 
articulation. Organization is concerned with how the content of the message is 
sequenced and how the ideas are related to one another. Content is concerned with the 
amount of relevance of information in the message and how the content is adapted to the 
listener and situation. Language is concerned with the grammar and words which are 
used to convey the message. 

Each of the four dimensions is rated on a four point scale: 1 is the lowest rating and 4 is 
the highest rating . A general set of principles underlies the rating scale for all four 
components. Ratings of 1 reflect speaking skills which are inadequate in meeting the 
requirements of the task. Ratings of 2 reflect speaking skills which are minimal in 
meeting the requirements of the task. Ratings of 3 reflect speaking skills which are 
adequate in meeting the requirements of the task. Ratings of 4 are superior in meeting 
the requirements of the task. 

Individuals who act as raters for the speaking assessment need to take the role of a naive, 
objective listener. The rater must be naive so that the rater can base his or her rating on 
exactly what the speaker says. The rater must be careful not to let his or her own 
knowledge and experience influence the rating. The rater must face each speaker as if it 
were a new experience. The rater must also be objective so that he or she does not let a 
particular set of norms of social acceptability influence the rating. The rater must 
evaluate the speaker in terms of how well the speaker meets the requirements of the 
speaking task, irrespective of the particular communication style the speaker uses. 



DELIVERY 



The delivery rating focuses on the transmission of the message. It is concerned with 
volume , rate and articulation . Articulation refers to pronunciation and enunciation. 
Some examples of poor articulation include mumbling, slurring words, stammering, 
s: .uttering and exhibiting disfluencies such as ahs, uhms or "you knows." 

1 = The delivery is inadequate in meeting the requirements of the task. 

e.g., • The volume is so low that you cannot understand most of the message. 

• The rate is so fast that you cannot understand most of the message. 

• The pronunciation and enunciation are so unclear that you cannot 
understand most of the message. 

2 = The delivery is minimal in meeting the requirement of the task. 

e.g., • The volume is too low or too loud. 

• The rate is too fast or too slow. Pauses are too long or at inappropriate 
spots. r 

• The pronunciation and enunciation are unclear. The speaker exhibits many 
disfluencies siKh as ahs, uhms or "you knows." 

• You are distracted by problems in the delivery of the message. 

• You have difficulty understanding the words in the message. You have to 
work to understand the words. 

3 = The delivery is adequate in meeting the requirements of the task. 

e.g., • The volume is not too low or too loud. 

• The rate is not too fast or too slow. Pauses are not too long or at 
inappropriate spots. 

• The pronunciation and enunciation are clear. The speaker exhibits few 
disfluencies, such as ahs, uhms and "you knows." 

• You are not distracted by problems in the delivery of the message. 

• You do not have difficulty understanding the words in the message. 

4 = The delivery is superior in meeting the requirements of the task. 

e.g., • The speaker uses delivery to emphasize and enhance the meaning of the 
message. The speaker delivers the message in a lively, enthusiastic 
fashion. 

• The volume varies to add emphasis and interest. 

• Rate varies and pauses are used to add emphasis and interest. 

• Pronunciation and enunciation are very clear. The speaker exhibits very 
few disfluencies such as ahs, uhms or "you knows." 



NOTE: In articulation you maybe concerned with accent. However, articulation should 
be rated with respect to your ability to understand the message, not the social 
acceptability of the accent. One particular accent is not considered better than another. 
REMEMBER, in this component you are rating how the student speaks, not what the 
siudmt says. 



ORGANIZATION 



The organization rating focuses on how the content of the message is structured. It is 
concerned with sequence and the relationships among the ideas in the message. 

1 = The organization is inadequate in meeting the requirements of the task. 

e.g., • The message is so disorganized that you cannot understand most of the 
message. 

2 = The organization is minimal in meeting the requirements of the task. 

e.g., • The organization of the message is mixed up; it jumps back and forth. 

• The organization of the message appears random or rambling. 

• You have difficulty understanding the sequence and relationships among 
the ideas in the message. You have to make some assumptions about the 
sequence and relationships of ideas. 

• You cannot put the ideas in the message into an outline. 

3 = The organization is adequate in meeting the requirements of the task. 

e.g., • The message is organized. 

• You do not have difficulty understanding the sequence and relationships 
among the ideas in the message. You do not have to make assumptions 
about the sequence and relationships of ideas. 

• You can put the ideas in the message into an outline. 

4 = The organization is superior in meeting the requirements of the task. 

e.g., • The message is overtly organized. 

• The speaker helps you understand the sequence and relationships of ideas 
by using organizational aid5 such as announcing the topic, previewing the 
organization, using transitions and summarizing. 



NOTE: Make sure you are not unconsciously "filling in" organization for a speaker, 
because you happen to know something about the speaker's topic. If you have to make 
assumptions about the organization, this fact should be reflected in your rating. 
REMEMBER, in this component you are rating how the student organizes the message, not 
what the student says. 



CONTENT 



The content rating focuses on the specific things which are said. It is concerned with the 
amount of content related to the task, the relevance of the content to the task and the 
adaptation of the content to the listener and the situation. 

1 = The content is inadequate in meeting the requirements of the task. 

e.g., • The speaker says practically nothing. 

• The speaker focuses primarily on irrelevant content. 

• The speaker is highly egocentric. The speaker appears to ignore the 
listener and the situation. 

2 = The content is minimal in meeting the requirements of the task. 

e.g., • The speaker does not provide enough content to meet the requirements of 
the task. 

• The speaker includes some irrelevant content. The speaker wanders off 
the topic. 

• The speaker adapts poorly to the listener and the situation. The speaker 
uses words and concepts which are inappropriate for the knowledge and 
experiences of the listener (e.g., slang, jargon, technical language). The 
speaker uses arguments which are self-centered rather than other- 
centered. 

3 = The content is adequate in meeting the requirements of the task. 

e.g., • The speaker provides enough content to meet the requirements of the task. 

• The speaker focuses primarily on relevant content. The speaker sticks to 
the topic. 

• The speaker adapts the content in a general way to the listener and the 
situation. The speaker uses words and concepts which are appropriate for 
the knowledge and experience of a general audience. The speaker uses 
arguments which are adapted to a general audience. 

4 = The content is superior in meeting the requirements of the task. 

e.g., • The speaker provides a variety of types of content appropriate for the 
task, such as generalizations, details, examples and various forms of 
evidence. 

• The speaker adapts the content in a specific way to the listener and 
situation. The speaker takes into account the specific knowledge and 
experience of the listener, adds explanations as necessary and refers to the 
listener's experience. The speaker uses arguments which are adapted to 
the values and motivations of the specific listener. 



NOTE: This rating is concerned with content in terms of quantity, relevance and 
adaptation . It is not concerned with content in terms of accuracy . Concerns with 
accuracy of content fall outside a speaking skills assessment. Also, make sure you are not 
unconsciously "filling in" content for a speaker because you happen to know something 
about the speaker's topic. If you add information, this fact should be reflected in your 
rating. REMEMBER, in this component you are rating the quantity, relevance and 
adaptation of what the student says, not the accuracy of what the student says. 
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LANGUAGE 



The language rating deals with the language which is used to convey the message. It is 
concerned with grammar and choice of words . 

1 = The language is inadequate in meeting the requirements of the task. 

e.g., • The grammar and vocabulary are so poor that you cannot understand most 
of the message. 

2 = The language is minimal in meeting the requirements of the task. 

e.g., • The speaker makes many grammatical mistakes. 

• The speaker uses very simplistic, bland language. The speaker uses a 
"restricted code," a style of communication characterized by simple 
grammatical structure and concrete vocabulary. 

3 = The language is adequate in meeting the requirements of the task. 

e.g., • The speaker makes few grammatical mistakes. 

• The speaker uses language which is appropriate for the task, e.g., 
descripti"^, language when describing, clear and concise language when 
giving information and explaining, persuasive language when persuading. 
Th. speaker uses an "elaborated code," a style of communication 
characterized by complex grammatical structure and abstract vocabulary. 

4 = The language is superior in meeting the requirements of the task. 

e.g., • The speaker makes very few grammatical mistakes. 

• The speaker uses language in highly effective ways to emphasize or 
enhance the meaning of the message. As appropriate to the task, the 
speaker uses a variety of language techniques such as vivid language, 
emotional language, humor, imagery, metaphor, simile. 



NOTE: In language you may be concerned with students who come from backgrounds 
where a foreign language or a non-standard form of English is spoken. However, language 
should be rated with respect to your ability to understand the message, not the social 
acceptability of the communication style. If a speaker's use of incorrect or nonstandard 
English grammar interferes with your ability to understand the message, this fact should 
be reflected in your rating. REMEMBER, in this component you are rating how the 
student conveys the message through language, not what the student says. 
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Attachment B 



MASSACHUSETTS DEPARTMENT OF EDUCATION ASSESSMENT OF BASIC SKJLLS 

ONE-ON-ONE SPEAKING TASKS 



Description Task : 

Think about your favorite class or extracurricular activity in your school. Describe 
to me everything you can about it so that I will know a lot about it. (How about 
something like a school subject, a club or a sports program?) 



Emergency Task : 

Imagine that you are home alone and you smell smoke. You call the fire department 
and I answer your call. Talk to me as if you were talking on the telephone. Tell me 
everything I would need to know to get help to you. (Talk directly to me; begin by 
saying hello.) 



Se quence Task : 

Think about something you know how to cook. Explain to me step-by-step how to 
make it. (How about something like popcorn, a sandwich or eggs?) 



Persuasion Task : 

Think about one change you would like to see made in your school, like a change in 
rules or procedures. Imagine I am the principal of your school. Try to convince me 
that the school should make this change. (How about something like a change in the 
rules about hall passes or the procedures for enrolling in courses?) 
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EXHIBIT 2. Speaking Objectives 
Mai;sachusetts State Department of Education 



A. BASIC ORAL COMMUNICATION SKILLS 

1. Use words and phrase*^ appropriate to the situation. (2.1) 

2. Speak loudly enough to be heard by a listener or group of listeners. (2.1) 

3. Spsak at a rate the listener can understand. (2.1; 

4. Say words distinctly. (2.1) 

B. PLANNING, DEVELOPING AND STATING SPOKEN MESSAGES 

1. Use words in an order that clearly expresses the thought. (2.2, 2.6) 

2. Organize main ideas for presentation. (2.5) 

3. State main ideas clearly. (2.8) 

4. Support main ideas witn important details. (2.5, 2.8) 

5. Demonstrate knowledge of standard English usage. (2.6) 

C. COMMON USES OF SPOKEN MESSAGES 

1. Use survival words to cope with emergency situations. 

2. Speak so listener understands purpose. 

3. Ask for and give straightforward information. (2.2) 

4. Describe objects, events and experiences. (2.2, 2.6) 

5. Question others' viewpoints. (2.3) 

*Number in parentheses indicate E nglish Language Arts Common Curriculum Goals for 
Oregon which are related to the Massachusetts objectives. 
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APPENDIX D 
SPEECH RATING SCALES 



EVALUATION FORMS* 



The following pages contain sample evaluation forms that may be useful in various kinds 
of classrooms. Their intended uses are described below. 

Introduction Rating Scale. This form is used for evaluating the introduction in a speech. 
The assignment is to present a one-to-two minute introduction to a speech. Even short 
assignments like this deserve to be evaluated. 

Speech Rating Scale. This scale is used to evaluate both informative and persuasive 
speeches. It was originally used at the University of Iowa, but has undergone extensive 
revision based on our research. The subquestions in each category serve as criteria for 
making the judgment about the category and provide helpful information to raters. 

Speech Rating Blank. This scale was developed and tested in a middle school classroom. 
Its strength is that it only asks for yes and no judgments. It should be noted that any of 
these forms can be used at the elementary and middle school levels. It is a matter of 
adapting the language and the criteria, rather than the entire form. 

Oral Interpretation Rating Blank. This scale was developed for use in evaluating ora! 
interpretation assignments. Note that it has more categories because it is used in r. 
special situation. Many people do not like to use numbers when evaluating oral 
interpretation, but our research shows that it has worked just as well as letter grades. 

Short Speech Feedback Form This form is shortened in that is contains fewer cues on 
how to evaluate each category. It has more space for written comments, however. This 
form is useful in later assignments when everyone is familiar with all the criteria in each 
category. 

Overall Comments 

It should be noted that any of these forms can be extended by using the back of the page 
for further comments. Comments are very helpful for the student, and the evaluator, 
whether student or teacher should be encouraged to use the back of any of these or other 
rating forms for extensive comments. 



*Taken from Bock, Douglas G. and E. Hope Bock. E valuating Classroom Speaking - ERIC 
Clearinghouse on Reading and Communication Skills, Urbana, 111.; Speech Communication 
Association, Annandale, Va., 1981. 
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INTRODUCTION RATING SCALE 



Speaker 



Date 



Comments 



Score 



Attention 

Does the introduction attract favorable 
attention? 

Are the attention factors obvious? 
Topic 

Does it state the topic clearly? 

It is obvious on which side of the topic the 
speaker stands? 

Is it clear where the speaker is headed with 
the topic? 

Credibility 

Does the intro establish initial credibility? 
Does the speaker use the sources of credibility? 

Delivery 

Is the speaker communicative? 
Was the eye contact direct? 
Here the gestures meaningful? 



Total 



10 

superior 



7 

average 



4 

inadequate 



poor 



Comment's: 



Total Points 
50-45 = A 
44-40 = G 



39-35 = C 
34-30 = D 
39-00 = F 



I.D.# 
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speech Rating Scale 



Speaker Date 



Subject ^ Assignment 



Items 


Comments 


Score 


ilfOQ ni T Q f'ir\n • i loot* QT*r*Qn<yomont' r\'f* iHooc? ^r^i^rf\/^^^^*i^^^\^n 
vyl 3^niZ.a LlUll. V^lcdr all all^CUlcIlL yJl iKlKzao i lIlLrvXiUCLlUIl, 

body, conclusion? Was there an identifiable pattern? 






Language: Clear, accurate, varied, vivid? Appropriate 
sianuaru ui uoa^c I 111 cuiivcr»ya Liuiidi mucic ; Wcrc 
unfamiliar * .rms defined? 






IVloLCllal. opcCli IC, VallLl, 1 ClCVallL, OUIIICICIIL, 111 LCI Co Llllg • 

Properly distributed? Adapted to audience? Personal 
credibility? Use of evidence? 






Delivery: Natural, at ease, communicative, direct? Eye 
contact? Aware of audience reaction to speech? Do 

p^pctiirpc matr*h \/r\ir*p JinH lantyiia<yp? 

^CoLUlL-o Ilia LCll V^lCC allU loll^Ua^C • 






Analysis: Was the speech adapted to the audience? Did the 
main noints ^iinnort thp niimosp^ 






Voice: Varied or monotonous in pitch, intensity, volume, 
rate, quality? Expressive of logical or emotional meanings? 








Total 


Scale: 





10 7 4 1 

superior average inadequate poor 



I.D.# 



Speech Rating Blank 



Name Date 



Subject of the Speech Teacher 

After you listen to each of the speakers, answer the following questions with a yes or a 
no. These questions will be used to help us decide which areas are important to us when 
preparing to give a speech, and how we may improve our speeches in the future. 

Organization — how the speech is put together or arranged. 

1. Could you easily pick out the main ideas of the speech? 

2. Did the speech have an introduction? 

3. Did the speech have a body? 

4. Did the speech have a conclusion or summary 

5. Was the speech developed or put together in a way that made it easy for the 
audience to understand? 



Language — the sentence structure of each speech. 

1. Were the explanations clear? 

2. Was the language easy to understand? 

3. Did the speaker make use of pauses to separate ideas from one another? 



4. Were there too many and's or uh's used? 

5. Was it easy to tell where one sentence stopped and the next one began? 



Material — what the speech was actually about. 

1. Was the subject interesting to i ou? 

2. Was the speech easy for you to understand? 

3. Did the speaker seem really to know the subject matter? 

4. Did the speaker seem comfortable and at ease while giving the speech? 



5. Was there eye ^ontact with the audience? 

6. Was the speaker aware of how the audience was reacting to the speech? 



7. Did the speaker make good use of gestures and body language? 

8. Did the main idea stand out above the other ideas? 

9. Were there other ideas less important but still necessary in the development of the 
speech? 

V' ^ce — how the speaker sounded. 

1. Was the speaker's voice pleasing to the ear? 

2. Was the pitch varied — that is, did it go up and down? 

3. Was the speaker loud enough? 

4. Did the speaker talk too fast? 

5. Did the speaker use good expression? 

Total Score (add the number of "y^s" responses) 



Grade 



ORAL INTERPRETATION RATING BLANK 



Reader Assignment Total Score. 



Item C nment Score 



1. Introduction: Captures attention, sets the scene and mood, 
given needed background, informative. 






2. Material: Of interest to the audience? Is it adapted to the 
reader, assignment, occasion, and audience? Proper cutting? 






3. Eye contact: Does the reader try to reach each me^iber of the 
audience? Is there too much dependence on the manuscript? Is 
there effective character placement? 






4. Articulati^^n and pronunciation: Is it clear, correct, slurred, 
muffled? Are there defective sounds? Acceptable standards of 
pronunciation? 






5. Facial expression: Appropriate, varied, adapted to the reading? 
Does it aid in expressing the emotions in the selection? 






6. Poise: Confident, at ease, personality pleasing moves easily, 
projected to the audience? Aware of audience reaction to the 
reading? 






7. Bodily action: Is the read.?r animated? Are posture, action, 
and gestures constructive or distracting? Are gestures used 
eiiecLiveiy, variea, suiieo lo conieni ano purpose: 






8. Vocal quality: Is it pleasant to listen to? Is there suffi- 
cient variety, projection, clarity? Acceptable volume? 






9. Rate and timing: Are rate and pauses varied and suited to 
content and purpose? Too fast or too slow? 






10. Content: Communicated author's intent as stated in the intro- 
duction? Was the content adequate to support the reader's goal 
as stated in the introduction? 







Scale: Total 

10 987654321 Score 
superior average inadequate poor 

Additional Comments: l.d.ff 
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SHORT SPEECH FEEDBACK FORM 



Name: 



Ion 



Item 



2 



3 



4 



Comments 



Clearly stated thesis 



Examples 

Direct support 
Clear explanation 



Eye contact 



Gestures 



Body position 



Eval'jator 



Grade 



Key: 



2 
3 
4 



Needs a lot of work. 
Doing well some of the time. 
Doing well most of the time. 
Excellent job— keep it up. 
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NAME 



.CLASS 



. DATE 



COMPREHENSIVE SPEECH EVALUATION 

Sheet for Teachers 



NAME. 

CLASS 

TOPIC: 



TIME LIMIT: 



DATE 



SCORE 



TIME TAKEN: 



TOPIC AND PURPOSE 

Topic significant and 
limited? 



Specific purpose 
established? — 



DELIVERY 

Mental Alertness 
Awareness of each idea 
presented? 



Obvious sense of 
communication? . 

Natural? 



Projects good will? 

Body 

Eye contact adequate? . 
Posture acceptable? 



Movement appropriate? 

Gestures effective? 

Voice 

Distinct? 



Vocal variety adequate^ . 

Rate appropriate? 

Pitch appropriate? 



Volume appropriate? 



Fluency adequate? . 



Vocalized pauses avoided? 



LANGUAGE 

Uses expressions appropriate to 
oral communication? 



World choice appropriate? . 

Conveys ideas clearly? 

Grammar correct? 



Pronunciaticr. correct? , 



Increases interest and impact? . 



CONTENT AND ORGANIZATION 

Introduction 
Gets attention? 



Gives needed information? 

Makes purpose clear? 

Relates topic to audience? _ 



Gains good will of audience? . 
Development 

Main points clearly stated? 

Organization 
Well-structureH'^ 

LoNr;*!!;^ SOUnd? 



Transitions effective? . 
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NAME 



CLASS 



DATE 



Supporting Material Conclusion 

Clear? _ Emp.iaslzes main points? 

Sufficient? Provides a note of finality? 

Logical? Ends with audience interest high? 

Interesting? 

Convincing? 



Visual aids effective? 
COMMENTS: 
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NAME 



CLASS 



OATi 



PEER EVALUATION FORM 



SPEAKER: TOPIC: 



Indicate your evaluation by placing an X in the appropriate box following each item. 



Excellent 


Above 


Atverage 




None 




Average 








5 


4 


3 


2 


1 



Attention-getting device 












Clear purpose statement 












Clear organization of ideas 












Effective use of language 












Interesting audiovisual aids 












Selection of main ideas 












Adequate summary 












Closing statement 













DELIVERY 



Volume 












Eye contact 












Vocal expression/tone 












Facial expression 












Poise/self-control 












Pronunciation/Articulation 













COMMENTS: What 1 liked most about your speech was 



If you could improve one clement of your speech. 1 would sujgest that ^,»ou try to 
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APPENDIX E 



CONVERSATION AND GROUP DISCUSSION 
EVALUATION SHEETS 



NAME 



CLASS 



DATE 



GROUP DISCUSSION PARTICIPANT EVALUATION 

(Ungraded) 

Identify the procedure your group followed and evaluate your group s performance by 
responding to the items below. Wnte your responses in your communication journal. 

1. ld€>ntify the leader in the group. Explain the process followed to designate 
that person as leader. 

2. Did the leader accept responsibility effectively? What was the leader's 
style— laissez faire. authoritarian, democratic? 

3. Who. if anyone, did not participate adequately in the preparation and discussion^ 
What specific efforts were made to get the person involved? What, in your opinion, 
caused the difficulty with their participartion? 

4. List specific actions people in your group took to create a cooperative effort 
toward getting taslo completed. Also, describe how you decided to divide the work. 

5. List specific instances in which members of your group made preparation or 
discussion more difficult or less effective. 

6. Comment on the cohesiveness of the group. List specific instances of consensus, 
compromise, cooperation, etc. 

7. if you had to work with the same people again, what would you do differently? 
Be specific. 

8. List the names of the members of your group, beginning with the strongest 
contiibutor and ending with the weakest contributor. 

L 

2 

3. 

4. 

5. 

6 

7. 
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NAME 



. CLASS 



.DATE 



{ Situatiofi 


Often 


UtuiiSly 


Seldom 


Never 


NA 


11. 1 help others understand me 
by explaining what I think, 
feel and believe 












12. I use various forms of 
nonverbal communication 
to express myself in 
conversations. 
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NA«IE CLW8 DATE 



CONVERSATION SKILLS 

Seif-EvaSuation Sheet 



Read the items below and check the response that best describes you in the situation. 
(NA = not applicable to me) 



Situation 


Often 


Usually 


Seldom 


Never 


NA 


1. When 1 don t understand in- 
structions, 1 say so. 












2. In conversation, I seem to talk 
more than other*: 












3. I let the other person finish 
talking before 1 reply. 












4. I find my mind wandering while 
others talk 












5. I Qsk questions to gain a better 
understanding of what others 
are saying. 












6. I pretend to be listening when 
1 am not 












7. I can usually detect dif- 
ferences between what people 
are saying and what they may 
be feeling. 












8. When speaking, 1 pay attention 
to feedback from others. 












9. My words come out the way 1 
want them to in conversation. 












10. 1 assume others know what 1 
mean. 













^ (continued) 

E-3 

SPEECH FOR EFFECTIVE COMMUNICATION / EVALUATION FORM 297 



nted by permission of Harcourt Brace Jovanovich, Orlando, Fla. 



o 

CO 



Name 



Date 



Class 



5 
J? 



EVALUATION SHEET FOR GROUP DISCUSSION 



Directions: List the appropriate names in the first column. Then, rank your classmates 
in each category, using the following symbols: + = excellent, V = adequate, and - = 
needs improvement. 



Parti: Group Leader 


Name 


Preparation 


Introduction 


Encouraging 
Participation 


Qarifying 


Timing 


Siunmsxizing 
















Parti Participants 


Name 


Preparation 


Level of 
Participation 


Clarity of 
Expression 


Open- 
Mindedness 


Courtesy 


Listening 
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